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ABSTRACT 



MAJIK/1 is a HyperCard introduction to periodical indexes; to the procedures for using 
the University of Maryland. College Park (UMCP). Serials List to locate specific articles; and 
to the arrangement of periodicals in the seven libraries of the UMCP Libraries System. 
Designed, developed, formatively evaluated, and revised according to an instructional systems 
design model, the program provides basic, individualized instruction for students to master 
prior to more advanced classroom instruction in strategies and methods of library research. 
The need for the program, specification of its target audience, and overall content of 
instruction were established by the 1988 UMCP Library Education Committee. Instructional 
materials and activities were developed by a design team composed of a content specialist, an 
instructional designer, and a computer programmer. Evaluation activities and materials were 
administered to 28 UMCP upperclassmen in two junior composition classes. Analysis of pre- 
and posttest data suggested that students profited from use of the program. Analysis of 
observational data indicated that students had virtually no difficulty with content, encountered 
some problems with navigation, and found the package useful and attractive. Students' 
suggestions for improvement provided a rationale for incorporating a navigational tutorial and 
other minor revisions into the package to make it even more valuable. An article describing 
the project and its results has been accepted for publication by r pll P ye ^ Researgh U\?mk i. 
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Every day, an average of 10,000 patrons uses the seven libraries of the University of 
Maryland College Park (UMCP) Libraries System, asking an average of 1,200 reference 
questions. Many of these questions, especially in the Hornbake Library's undergraduate 
facilities, come from upperclassmen and are focused on finding periodical articles and then 
locating the appropriate periodicals within the UMCP Libraries System. Despite a one-hour 
library instruction session included in most freshman composition courses, it is thus evident 
that many transfer students and other upperclassmen have not mastered the skills of 
identifying and locating articles-skills that are required to complete library research 
assignments. 

MAJIK/ 1 --Magazine And Journal Instructional Kit. Part 1- was developed as a first step 
in addressing this need. The most basic of an anticipated four modules on locating periodical 
articles, the package is an outgrowth of the work of th.j 1988 Library Education Committee 
(chaired by Jackson and served on by Neuman) that identified the need for and general 
content of instruction at each of four levels of complexity. MAJIK/l-an explanation of 
periodical indexes in general, description of basic indexes, and instruction in locating articles 
within indexes and periodicals in particular UMCP libraries-provides a general overview of 
index use and sets the stage for instruction in the use of broader and more sophisticated 
periodical research materials, such as union lists and OCLC. Designed both to be used in 
conjunction with UMCP junior composition courses and to stand on its own as a self-paced 
module for individuals seeking to review nnd update their skills. MAJIK/l will be housed both 
in the microcomputer lab in the English Department and in the projected microcomputer 
facilities of the Hornbake Undergraduate Library. It will be publicized with handouts and 
referrals at the Hornbake Library's Information Desk. 

The creation of MAJIK/ 1 followed an eight-step process based on the principles and 
procedures of instructional systems design as explicated by Gagne and Briggs (1979), Dick and 
Carey (1985). Kemp (1985). and others. The first six steps-needs analysis, learner analysis. 
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specificafon of goais and objectives, development of test items, selection of th,s delivery 
medium, and development of materials and activities-comprised the design phase. The final 
two steps-formative evaluation and revision-comprised the evaluation phase. Exhibit 1. 
Budget Report, displays the final accounting for the funding received from the Council on 
Library Resources Cooperative Research Grant Program to conduct the project. 

DESIGN OF MAJIK/1 

Step 1: Needs Analysis 

Both general and specific needs were identified through the work of the Library 
Education Committee during the spring and summer of 1988. Members of this Committee 
included the UMCP Libraries Coordinator for Instructional Services, a faculty member of the 
College of Library and Information Services who teaches instructional systems design, two 
UMCP librarians who teach in the Library Instruction Program, and a faculty member of the 
UMCP English Department representing junior composition teachers whose courses include a 

session on Jibrary instruction. 

The Committee was charged originally with improving ?he junior component of the UMCP 
Library Instruction Program, which has been plagued by students' inability to profit from 
instruction in the advanced concepts included in this curriculum. Because many of these 
students have not mastered the basic concepts covered in the freshman component, junior 
instructors have been forced to spend time covering the basics and left unable to address such 
higher-level concepts as formulating successful search strategies. 

Early in the development process, the Committee focused on analyzing and improving the 
freshman component, considering that effective instruction at this level was prerequisite to 
effective instruction for juniors. The Committee soon recognized, however, that the 
instructional problem to be addressed did not necessarily involve the freshman program itself. 
Instead, the failure of many juniors to be ready for more advanced instruction seemed to stem 
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in large part from other causes. Because transfer students and students who place out of the 
freshman composition requiremcsnt do not necessarily encounter the basic concepts covered in 
the freshman component and because other students do not necessarily use those concepts 
until two years after instruction, many juniors' lack of appropriate background is clearly 
extraneous to the quality of the freshman component. Through the Committee's deliberations, 
it became evident that the primary need was not the revision of the freshman program but the 
development of an intermediate instructional experience for student*, who. for whatever 
reasons, needed basic instruction before advanced instruction was possible. 
Step 2: Learner Analysis 

Committee deliberations also revealed the characteristics of the target audience for 
whom the instruction would be designed: juniors taking the UMCP's required upper-level 
composition course, which includes a requirement for a paper involving library research. The 
audience would be primarily transfer students, students who had placed out of the required 
freshman composition course, and students who wanted to refresh the skills they had been 
taught in that course. This wide range of students for whom the materials might be assigned 
precluded any finer delineation of learner characteristics and suggested that the materials had 
to be developed at a very basic level to make them accessible to a broad audience. 
Step 3: Specification of Goals and Objectives 

Committee members with experience with the upper-level course analyzed the results of 
students' assignments and tests and drew upon their instructional experiences to identify the 
concepts students should master before more advanced instruction can begin. Extensive 
Committee discussions thus provided the raw data for this third phase of the project, which 
was the first (,o be accomplished under the CLR grant. Exhibit 2. "Objectives for HyperCard 
Project." displays the fnur categories of goals and objectives determined by the principal 
investigators in this phase. This list provided the basis for developing both the instructional 
and the evaluation materials for the project. 
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Step 4: Devebpineiit of T<^t Items 

Test items based on the list of goals and objectives were devised prior to the formative 
evaluation of the materials. Exhibits 3 and 4, -MAJIK/1 Formative Evaluation Pretesf and 
"MAJIK/1 Formative Evaluation Posttest; display the format in which the test items were 
presented to students in an attempt to determine whether learning had occurred as a result of 

exposure to MAJIK/1. 

The purpose of a formative evaluation is to gain information to guide the revision of a 
nc^ly developed instructional package. Because a formal analysis of learning gain is not 
essential to achieving this purpose, it had not been proposed as part of the original grant 
application. Nevertheless, as the project proceeded, the principal investigators concluded that 
conducting such an analysis might provide additional information upon which to base the 
revision of the materials. The design and implementation of a simple pre- and posttest 
strategy was therefore added to the proposed scope of work. As is common in the design of 
such a strategy, the posttest is a simple reorganization of the questions presented in the 
pretest. 

Step 5: Selection of the Delivery Medium 

Several considerations merged to suggest Apple's HyperCard as the most appropriate 
delivery medium for MAJIK/1. Because not all students need this elementary instruction, the 
Committee decided that an independent, stand-alone instructional module would provide the 
best solution to the identified instructional problem. Students would be required to complete 
this module outside of the limited class time available for library instruction, which could then 
be devoted to the advanced instruction the junior program is designed to deliver. Both the 
content of the module and ite implementation format cou'd thus alleviat3 the problems faced 
by library instruction staff charged with providing instruction in research methods and 
strategies but prevented from addressing these concepts by the lack of appropriate background 
possessed by the studente they encounter in class. 



Although several independent learning formats were considered, ultimately Apple's 
HyperCard was selected for several reason?. First, HyperCard incorporates the advantages of 
traditional computer-assisted instruction (CAI) for independent learning: individualization, 
self-pacing, immediate feedback, and flexibility in scheduling. Second, HyperCard expands 
upon the individualization irherent in CAI by offering rapid, flexible movement through 
information according to each individual's preferences and needs. Third, the UMCP Libraries 
System had recently purchased for Hornbake Library-the library most frequently used by the 
target students-hardware capable of running HyperCard programs and was eager to obtain 
appropriate software for the machines. Finally, the desire to explore the possibilities of this 
exciting new format as they relate to library instruction offered a particularly appealing 
development challenge. In sum, the needs identified in the earliest stage of the project 
merged with a variety of additional factors, both theoretical and practical, to govern the 
selection of the medium. 

Step 6: Development of Materials and Activities 

The bulk of activity during the grant period involved the development of the MAJIK/i 
materials themselves. These materials are arranged in the four major components outlined in 
the list of goals and objectives described above: "Introduction to Periodical Indexes," "Using 
Periodical Indexes," "Using the I MOP Serials List," and "UMCP Libraries." Each component 
includes the presentation of the appropriate contem and instructions for navigating in a 
HyperCard environment; each of the two middle componems, which carry ihe major portion of 
the instruction, includes a series of exercises designed to enable students to practice using the 
concepts presented. 

The package takes advantage of the HyperCard environment through a structure that 
allows students flexibility of movement within and across components: within components, 
students may move freely among the definitions and explanations according to their individual 
needs for information. Across components, students may choose to go through one, two, 
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three, or all four segments of the package and may go through them in any ordei they prefer. 

Development of the materials involved the close collaboration of the three individuals 
who comprised the instructional design team: Jackson, who provided subject matter expertise 
as well as experience in library instruction and who was responsible for writing the content of 
tiie package; Neuman, who provided instructional design expertise and experience in designing 
computer-based instruction and who was responsible for editing die content and structuring it 
for presentation in an electronic instructional format; and Rebekah Zanditon, tiie consultant 
funded through tiie grant itself, who was responsible for programming the materials and whose 
experience as a student ia the M.L.S. program in the UMCP College of Library and 
Information Services enabled her to bring -,ontent knowledge as well as programming expertise 
to her roie. 

The development followed the iterative process typical of instructional systems design 
projects: content for each component was written, reviewed, revised, programmed, »-eviewed. 
and revised again. Over the course of Uie effort, approximately 230 HyperCard screens were 
created: 80 containing primary instructional information and 150 containing definitions, 
answers, and similar supplementary matarial. Exhibits 5 tiirough 10, "Exemplary MAJIK/1 
Screens," display the format and content of representative screens. A number of mechanisms 
were incorporated into the program to enable the students to navigate tiirough the material in 
a flexible manner. Boldface terms and phrases provide the mechanism by which users can 
move to definitions of key concepts. Buttons provide tiie mechanism by which users can self- 
check their answers to the questions in tiie practice exercises. The icon of tiie "1" is the 
mechanism by which users can return to the beginning of tiie program, while tiie icon of the 
question mark provides the package's "help" mechanism. At almost any point in MAJIK/1, 
students may leave one part of the program to go to another simply by clicking on an icon 
designed specifically to represent the destination. 
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EVALUATION OF MAJIK/1 

Step 7: Form«llve Ev«lu«tlon 

The materials themselves were completed by early June in anticipation of a formative 
evaluation to be conducted during the two summer sessions offered by UMCP. Human subjects 
clearance was obtained (see Exhibit 11) and instructors in the English Department were 
contacted in order to idemify individual classes that would be involved in the evaluation 
effort. Three such classes were identified--one to be taught in the first session and two to 
be taught in the second. The Information Processing Laboratory of the College of Library and 
Infoimation Services (CLIS) was selected as the evaluation site, and MAJIK/1 was installed on 
two Macintosh SE's there so that students could come individually or in pairs for the 
observation phase. 

The class ^ught in the first summer session was viewed as a pilot for the evaluation 
processes and instrumentation, and the minimal amount of data collected in that effort was 
not analyzed. Logistical and other difficulties were resolved during this first session so that 
the full evaluation effort was ready to begin in July, as specified in the grant proposal. 

Materials developed for the formative evaluation include the pre- and posttests described 
above, a "How to Work MAJIK" sheet (Exhibit 12). and an Observation Protocol (Exhibit 13). 
&ndy Dwiggins. a CLIS Ph.D. student, was identified as the graduate assistant to be funded 
through the project and trained in the strategy that constituted the core of the data 
collection effort. This strategy-interactive observations of users of the materials-was 
derived from tenets and procedures of naturalistic inquiry as described by Guba (1979. 1987). 
Lincoln and Guba (1985). and Neuman (1989). 
Data Collection 

Dwiggins. Jackson, and Neuman met with the students in both classes-explaini.-Jg the 
project, requesting the students' participation, and administering the pretest. Students used 
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the materials between July 28 and August 3 while Dwiggins observed their efforts and 
questioned them about their perceptions eid behaviors. As the final step in the evaluation 
process, students took the posnest during their regularly scheduled classes on August 8. 

Students participated in the interactive observationc in the Information Processing Lab of 
the College of Libiary and Information Services (CLIS). Special arrangements were made to 
allow access of the students to this site, whicl. is reserved for the use of CLIS students. An 
observation schedule was established to ensure Dwiggir.s' presence while students used the 
MAJlK/1 materials. Dwiggins interacted freely with the students during the observations- 
interrupting their progress to ask them about problems they were encountering, questioning 
them when they finished individual sections of the program, and interviewing them upon 
completion of the entire experience in order to gain their insights about strengths, 
weaknesses, and necessaiy and desirable improvements. 
Data Analysis 

Data analysis proceeded along two fronts: statistical treatment of the pre- and posttest 
scores and naturalistic assessment of the observational data. The combination of both 
approaches was chosen in order to amass the greatest amount of information to guide the 

revision of the materials. 

StnH^nK a nd context . Twenty-eight students (17 male, 11 female) from two UMCP junior 
composition courses participated in the evaluation effort. At least half the participants had 
had previous experience with a Macintosh: nine described themselves as Macintosh users, 
while five said they were novices at using the Mac. Eight students reported no familiarity 
wjth the hardware and six did not provide this information. 

Time devoted to the program ranged from extremes of 19 to 50 minutes with a modal 
t:me of 35 minutes spent by nine of the students; half the participants completed the program 
in 32 to 38 minutes. There is no clear temporal pattern distinguishing those who were 
familiar with the Macintosh from those who were not: students at both extremes were self- 
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described Mac users, whiie those in the middle range included users, novices, non-users, and 
students for whom there is no data. 

While the 32- to 38-minute rwige suggests a broad estimate of the time required to 
complete the program, this figure is itself inflated: it reflects time spent responding xo 
questions interposed by the observer as well as time spent working on the program itself 
Indeed, even the extremes are likely to be artifacts of the research setting, reflecting one 
student's extreme facility with the tool and the other's interest in exploring the design of the 
program itself. 

A^^IV^ H "in . Pecause the evaluation was designed to glean information to 

guide the revision of the program rather than to establish the effectiveness of the materials, 
no rigorous statistical procedures were followed. The students involved in the process were 
not chosen randomly, for example, and no control group was involved in the pre- and 
posttesting. Nevertheless, in an attempt to draw some general iaferences about the 
instructional effectiveness of MAJIK/1. Dwiggins and Neumar. used the microcomputer version 
of SPSS to analyze basic differences between pre- and pcsttest -cores for the 24 students for 
whom such data was available. 

This analysis revealed a notable increase in student performance between the pre- and 
the posttest. Scores on the pretest ranged from 31 to 80 percent correct, while tiiose on the 
posttest rariged from 55 to 83 percent. The mean score increased from 59.2 percent correct 
on the pretest to 68.4 percent correct on the posttest-an increase of approximately 15%. 
While these figures cannot be taken as proof of the program's effectiveness, they at least 
suggest that MAJIK/1 has proraising instructional ^alue. 

^.,iv.i. of observ ation^dJiaia. The Observation Protocol was designed to structure both 
the collection and U»e analysis of data according to the four major segments of the MAJIKL/l 
materials described above. In addition, the observations themselves revealed another dimension 
*o be addressed: the learning and use of the features specific to HyperCard that allow users 
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to navigate HyperCard materials with flexibility and ease. 

Because of the small number of students observed using MAJIR/l (28). manual rather 
than electronic means were used to analyze the data. Preliminary data analysis consisted of 
creating matrices that displayed the data for each student according to each of the categories 
in the Observation Protocol (see Exhibit 14). Final analysis involved summarizing this 
information by category (including the new "HyperCard" category revealed during the 
observations); compiling students' nummary comments; and drawing the conclusions and 
implications reported below. 

1. t^trfyl r^'"" ^" Hvn prr.rd Features. The observations revealed that students had 
more difficulty mastering thz procedures required to navigate MAJIK/1 than they had 
mastering its content. While this finding is not surprising, give-, the novelty of the format 
for many of the student participants, it suggests an area of concern both for MAJIK/l's 
designers and for the designers of similar materials. In order to accommodate student groups 
who will undoubtedly include learners who are unfamiliar with the terminology, conceptual 
structure, and navigational possibilities inherent in HyperCard, it is clear that designers of 
HyperCard packages must incorporate features that will enable such rtudents to use the 

packages independently. 

Throughout their use of the program, students* problems navigating HyperCard were 
basic. Initially, for example, several students were unfamiliar with the notions of "clicking." 
"buttons." and "icons" and had to be instructed by the observer in these ideas and in their 
relationship. A quarter of the students e.xpressed dismay about the amount of navigational 
material-particularly icons-they were expected to know in order to use the program: "We're 
supposed to remember all these?" A number of students also failed to remember that clicking 
on a boldface item would lead to its definition, and several made navigational errors that can 
be attributed to their lack of understanding of how to move through HyperCard material: 
"How do you get back to the question mark [i.e., the help] section?" 
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2. | ntrpHu<.tinn to Pe riodical Indexes. Students had virtually no problems with the 
content pwsentcd in this brief component, which de^'ines periodical indexes; differentiates 
between general and specialized indexes; and explains the various formats (print, microform, 
and electwnic) in which indexes are published. Some of the navigational problems described 
above, however, began to appear in this segment of the package. 

Several students chose to skip this section entirely, but most proceeded through it (and 
the rest of the program) in an essentially linear manner. While the straightforward nature of 
this segment suggests that linear progression is the most appropriate app'^ach here, 
consideration of how the segment and the package as a whole might take better advantage of 
the flexibility inherent in HyperCard is probably warranted. 

3. iTcjn B periodical Indexes . Students' use of this most extensive component of MAJIK/1 
revealed few difficulties with content but a number with navigation. Although the content 
here is the most complex in the package-choosing appropriate indexes; identifying the 
elements of citations; and using headings, subheadings, and cross references-students seemed 
to understand it readily. The fact that a number of students reported that they had known 
the content beforehand, however, means that the question of its clarity to all members of the 
eventual target audience (i.e., students for whom the materials will be an introduction rather 
than a review) was not adequately answered during the evaluation. 

Students seemed especially pleased with the practice exercises introduced in this segment, 
giving evidence of active <mgagcment (e.g., nodding their heads as they proceeded) and 
offering such positive comments as "good," -helpful," and "There were enough exercises to 
make you feel comfortable with the material." The number and tenor of students' comments 
on the exercises suggest that they are a useful and attractive feature. 

Several students found the directions for completing the exercises unclear. The primary 
source of students' confusion seemed to be the design of the exercUes rather than the 
directions themselves. As illustrated in Exhibit 7, the exercises are a series of multiple-choice 
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questions witJ> lettered options and with an instruction to answer each question mentally and 
then click on a button to confirm the correctness of the answer. Almost half the students, 
however, initially tried to enter the letters of their choices-expecting the program to accept 
and judge th,ir responses in the now-familiar format of comparable computer-assisted 
instruction tasks. Apparently beca.se the MAJIK/1 format ran counter to their expectations, 
these students were initially disconcerted by it. Students overcame their confusion readily, 
however, suggesting that the problem is a minor one. 

Most students proceeded through this segment in a linear fashion, although a number 
traversed the material mon flexibly. Of these, six discovered the strategy of using the table 
of contents as a focai point and employed this approach successfully. A number of student, 
forgot the option of using boldface to move to definitions, while several f.iled to realize that 
^licking on a boldface italic term or phrase (i.e.. the title of an index) would lead them to an 

explanation of its nature and use. 

4. ^uin p thft UMCP Serials List . Students' perceptions and behaviors related to this 
component paralleled their experiences with the previous one. Students encountered few 
problems with the content, which includes an explanation of the microfiche reader and 
instructions on how to select and use the microfiche cards that comprise the UMCP Serials 
List to locate periodicals stored in various formats in the UMCP Libraries (i.e.. microform, 
bound volumes, current stacks). Students used the practice exercises extensively and actively 
and made few suggestions about improving this segment of MAJIK/1. Almost all students 
reverted to a strictly linear navigational strategy here, a situation that seems to reflect the 
nature of the content presented in this component. 

5. iTMrP Libraries . This segment-a simple listing of the UMCP Libraries and the way 
each houses its periodicals-presented neither conceptual nor navigational difficulties for 
students. The nature of the information included in this component suggests that the 
capability for getting screen prints would be a useful addition to the program here, as it 
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might be for other segments as well. Several students failed to realize they had finished the 
program, suggesting that the revision of the end card here would also be helpful. 

6. flvm r^rv romments . Upon completion of the program, students were asked to 
summarize their MAJIK/1 experience, anil abnost all their summary comments were positive. 
Students focused on such features as the program's good graphics (13 comments), clear text 
(10). helpful practice exercises (9). self-pacing (7). and ease of movement within the package 
(5). Selected comments in each of these categories are displayed in Exhibit 15. Students' 
general comments focused on the informative, useful, easy-to-use. and creative nature of the 
materials, with one student commenting on the good balance between text and graphics and 
another pronouncing the program "very worthwhile." One student noted that "(IJ learned as 
much from the module as I had in class." 

Nine students in the group explained that the materids had been a review for them, 
while the others did not offer that information. Several suggested that the package would be 
good for freshmen, sophomores, and transfer students: one recommended that it be 
compulsory for freshmen and another wished it had been available "when I was a freshman." 

Not all summary comments were positive, of course: five students criticized the amount 
of detail in the program, five complained that the icons were difficult to remember, three 
protested the amount of time it took them to complete the package, and two noted the 
confusion they had experienced over the use of boldface. As one critic noted. "The only 
reason a person would go through all this would be if the professor made them." 
St«p 8: Revision of MAJIK/1 

Both statistical and naturalistic analyses of student performance suggested tha: the 
package as originally designed effectively met the needs it set out to address. In particular, 
students' comments and observed behaviors indicated that the materials were attractive, useful, 
and reinforcing. Having achieved a measure of success even in its initial trial. MAJIK/1 as 
revised should be even more helpful and worthwhile. 
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The revision process involved analyzing the data for suggestions for revision and 
incorporating those suggestions that would most enhance the final product. Most of the 
revisions addressed general and specific devices that will enable all students-even those 
unfamiliar with HyperCard-to proceed through the materials independently. For example, the 
icons were examined to determine whether they could be made more representative. Labels 
were added to the icons to enhance their visual stimulus with a verbal one. Hints and 
prompts about the meanings and uses of the various navigatiomil features were incorporated 
directly into appropriate screens to serve as reminders throughout the package. 

Most importantly, an option?! tutorial explaining the features of the program and 
suggesting navigational strategies was included at the beginning of the package to provide the 
support students clearly need. The tutorial includes not only instruction in the meanings and 
uses of navigational features but both graphic aad verbal information that should enhance 
students' abilities to proceed independently. An overview of the entire package ai well as an 
overview of its most complex component should help students develop an appropriate 
conceptual structure to guide their progress through the materials. Instructions in using the 
table of contents as a focal point, or pivot, should give students a way to orient themselves 
by choosing individual segments and returning regularly to a recognized "home base." 

CONCLUSION 

The design, development, evaluation, and revision of MAJIK/1 have resulted in several 
products of use to library instructors. Most obviously, of course, is the program itself, which 
can be used in the UMCP Libraries System to address the instrucUonal needs of many transfer 
and other upperclass students who must master basic concepts before they can profit from 
advanced instruction in library research skills. Secondly, publications and presentations 
related to the product as well as to its development process will convey the knowledge and 
expertise developed through this project to faculty and sttff beyond the UMCP campus who 
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are exploring the application of computer assisted instruction, and particularly HyperCard 
instruction, to basic library education. And third, the combination of an instructional systems v 
design (ISD) approach to create the materials with the use of naturalistic inquiry (NI) ^ 
techniques to evaluate them has provided the basis for a development model »hat is not only 
new to library education but whose use is in its infancy in other fields as well. 

It is this ISD/NI model that is potentially the most significant result of the MAJIK/1 j 
project. Combining the advantages of insights from two separate academic disciplines, the ^ 
model provides guidance for the entire development cycle for library education materials. j 
Following this systematic and straightforward approach can result in materials that effectively 
meet the needs of the students for whom they are designed. : 

In the design phase for MAJIK/1. the use of ISD techniques resulted in well-grounded 
analyses of instructional needs and learner characteristics, in the precise specification of 
relevant goals and objectives, in the writing of test items directly related to those goals and ^ 
objectives, in the selection of the delivery format that was most appropriate for both the 
content of the instruction and the needs of the target audience, and in the thoughtful ' 
development of materials and activities designed to address the goals and objectives for the J 
identified learners. Moreover, the "team" approach common to ISD efforts led to the , 
collaboration of experts in three areas-library education, instructionnl message design, and 
HyperCard programming-to ensure that all appropriate areas of expertise were tapped as the 
materials were designed. The iterative process characteristic of the ISD approach guaranteed 
that all three developers would review and comment upon one another's efforts, ensuring that 
the prototype product reflected the best insights of the group as a whole. 

Fcrmative evaluation and prototype revision are inherent in the ISD approach, but 
employing NI techniques during evaluatior in order to get optimal information for revision is a 
relatively new strategy. Several authors have suggested an ISD approach to designing 
bibliographic instruction (for example, Cottam & Dowell. 1981; Jacobson & Albright. 1983; 
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Kenny & McArthur. 1984; Miller & Bratton. 1988); but none has sought to combine that 
approach with an Nl-based evaluation. Drawing upon the assumptions and techniques of Nl in 
this project, however, brought important strengths to the MAJIK/1 materials. Of obvious 
value are the insights reU .d to specific revisions gleaned through the interactive observations 
of participating students. Even more significant, however, is the discovery through this 
technique of the major deficiency in the program-i.e.. the lack of adequate support for 
students to draw upon to navigate the HyperCard materials successfully. Thus, while the more 
traditional pretest-posttest approach yielded little information for revision (and none on this 
important topic). NI provided insight into the revisions that are most likely to enhance the 

quality and utility of the materials. 

The hallmark of NI is its underlying assumption that the individuals in a research study 
are "participants- with valuable insights to offer rather than -subjects- to be tested according 
to the researcher's own preconceptions and beliefs. The NI strategies used in this study 
capitalized on this assumption, ensuring that individuals like those for whom the materials 
were designed had ample opportunity to react to the materials, to identify effective and 
ineffective instructional approaches within them, and to suggest ways to improve the product. 
Clearly, incorporating the information gained through this approach into the revision of 
MAJIK/I should result in a product that is even more effective. 

The creation of MAJIK/1 thus provided a test case for the combination of the techniques 
of ISD and of NI into a new model for developing and testing an interactive instructional 
package for basic library education. While thr resulting product is by no means perfect and 
while difficulties in implementing this combination of techniques remain, the MAJIK/1 
experience suggests that the basic model itself offers a feasible approach to similar 
development projects. And because this project has served as a preliminary validation of the 
model, it can be used in similar projects with some confidence that the resulting materials will 
be effective with and attractive to the audiences for whom they are designed. 

ERIC J n 



EXHIBIT 1 
BUDGET REPORT 
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EXHIBIT 2 
OBJECTIVES FOR HYPERCARD PLIOJECT 

Introduction to periodical indexes 

Thy vrr "'ill <1ff^?rifrt thT «'"<^ numoses of indexes. 

A. The user will define a periodical index. 

B. The user will name two different types of periodical indexes, 
e,g., newspaper, magazine, mixed. 

C. The user will differentiate between a general and a specialized 
index. 

D. The user will state the titles of three different periodical 
indexes. 

E. The user will state that print, microfilm, and electronic 
indexes are available. 

. Instruction in the use of periodical indexes 

7> y yficr will selc f t pnnrnnriate jpdexfiS and Stihiftfft hMdtng? and 

l ^ ^ntifv the element s ftf a fetation in n hmQ nenffral i noffX. 

A. Given a particular topic, the user will select an appropriate 
general index for that topic. 

B. Given a particular topic, the user will locate appropriate 
subject heading(s) within an index for that topic. 

C The user will identify all the elements in a given citation, 
i.e., arUcIe author, article title, periodical, title, volume, 
date, pages. 

D. The user will match abbreviated journal tiUes to the full titles. 
Tfi f ^...r will use su M ff^r hidings. snhheadinRs. citntions . and cross 

r^fffrynces in index entries. 

A. Given a page from an index, the user wUl identify the subject 
heading, a subheading, a citation, and a cross reference. 

B. The user will follow a cross reference to its referent. 
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Instruction in the use of the University of Maryland, College Park, 
Serials List. 

ITl ff ^S*^*- will understand the oDe r gHnn of the microfiche mach i ng, 

A. The user will turn on the microfiche machine and insert the 
microfiche card properly. 

T),y Yfjfff u,}]} i^jg ntifv locations {inrt hnldines for anv periodioaL 
wjthin the UMC P Libraries. 

A. Given a periodical index citation, the user will identify the UMCP 
Serials List as the source of information to locate that article. 

B. Given r specific periodical title, the user will select the correct 
microfiche to be used. 

C. The user will identify and state the purpose of the index at the 
top of each column of the microfiche card. 

D. Given a specific periodical tiUe, the user will use the index to 
locate that title on the card. 

E. Given a specific periodical title, the user will determine the 
call number for that title. 

F. Given a specific issue of a periodical, the user will determine 
which library /libraries hold that issue. The user will identify 
what format that issue is in, i.e., microfilm, microfiche, bound, 
current stacks. The user will identify the appropriate strategy 
for finding a periodical title not found on the Serials List, i.e., 
asking the Reference Desk for assistance. 

DescripUon of the arrangemeuts of periodicals in the various UMCP Libraries. 

y g yr will deterriine wh f r^ ^^e Li^pi"fts s/he can find the 
Periodical needed. 

A. The user will explain the difference in the ways periodicals are 
stored in McKeldin and Hornbake Libraries. 

B. The user will identify the appropriate strategy for finding periodicals 
in the other UMCP Libraries, i.e., asking at each library how 
periodicals Are stored. 



EXHIBIT 3 



MAJIK FORMATIVE EVALUATION 
PRETEST 



Name: 



Student ID 



1. What is a periodical index? .^4.4„s«.„ 

a. A set of numbers measuring periodic activity 

b. A guide to the coa'cents of periodicals 

c. An organized group of periodicals 

d. None o£ the above 

2. Which of these are Kinds of periodical indexes? 

a. Newspaper indexes 

b. Magazine indexes 

c. Neither of the above 

d. Both of the above 

3. A general periodical index includes: 

a. Popular magazines 

b. Professional journals 

c. Both of the above 

d. Neither of the above 

4. A specialized periodical index includes: 

a. Popular magazines 

b. Professional journals 

c. Both of the above 

d. Neither of the above 

5-7. Name three periodical indexes. 



a. 



b. , ■ 

c. , . — 

8, Periodical indexes are available in these fonns: 

a. Print 

b. Microform 

c . Electronic 

d. All of the above 

9. Which index would you use first to find articles on school 
finance? 

a. Business periodicals Index 

b. National Newspaper Index 

c. Education index 

d. Magazins Index 
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10- Which subject heading(») would you select to find articles 
on the university of Maryland? 

a. universities— private 

b. Acade»ic achievement— secondary 

c . Sports— collegiate 
d- None of the above 

11- 20. Match the letters on the illustration with the correct 

names on the list below the illustration. 




Inditns of North Anserica 

See alsch 
Alcohol and Indians (Americtn) 
Cahuilla Indians 
Chippewa Indians 
Cree Indians 

Relislon and mylholoey^C. 

See also 
Totem poles 

cl^^'T»f"^inft native spirilualily: a call lo justice [cover 
sloM J. Magnuson. il Christ Cvniury 104:1114-17,6 





Citation 
subject heading 
subheading 
volume number 
date of publication 
author 

article title 
journal title 
page numbers 
cross reference 

21. The word "see" in an index tells you: 

a The index has no articles at all about this topic, 
b'. The index has no articles about the topic here but has 
related articles under another heading. 

c. The index has articles here and under another heading. 

d. None of the above. 

22 The words "see also" in an index tell you: 

22. The articles at all about this topic. 

b. The index has no articles about the topic here but has 
related articles under another heading. 

c. The index has articles here and under another heading. 

d. None of the above. 
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23. What thoulrt you use to learn which UMCP library or libraries 
havt a periodical you need? 

a. UMCP Serials List 

b. A periodical Index 

c. A campus nap 

d. GEAC, the caiipus online catalog 

24. TO find tTiie .TQurnul of Special -Education Technology in the 
UMCP Libraries, you would look:. on a microfiche with which 
heading? 

a. Jou rnal of Science Education 

b. Jour nal of space. Exploration 

c. special Education Abstracts; 

d. special Education, Journal of 



25-27. 



Match the letters on the illustration with the correct 
nanes on the list below the illustration- 




28. 



( ) index 
( ) frame 

( ) identifying letter 

in which frame in the illustration below would you find the 
journal Scottish Poetry ? 



GRID MDCX 



SCitllTIFIC AMEtlCAW (ARCHITECTS AIR) BUILDERS EDIT 

SCiENTVIC Al» TCCMHICAL SOCIETKS OF CAIIADA 

SCCNXf tC PAKRS OF THE ■iStlTOTE OF PHYSICAL AHO 

SCI^flCMMWS , 
SCOTTISft^AliTtEVEV 

SCOTTISH POCtRY 



a. Frame L 

b. Frame 15 

c. Frame L 15 

d. Any of the above 
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29-32. Match the letters on the illustration with the correct 
names below the illustration. 



HOOERH TEXTILES 
^<-.TSt688.A1R282 FOLIO 
MCKELDIN LIBRARY 
bf-STACKS: V. 50(l969)-62(No. 1-9.1981) 

UNDERGRADUATE LIBRARY 

PER. STACKS: V. 50(1969)-54(1Q73). 
PER. STACKS: mCROFILH->d 

V. 55(1974)-62C1981) 




) call number 

) library where periodical can be found 

) part of library where periodical can be found 

} format of periodical 



33. in McKeldin Library, periodicals are arranged like this: 

a. Bound volumes are in the stacks; current issues are 
in the Periodical Room. 

b. Both bound volumes and current issues are in the 
Periodical Room. 

c. Both bound volumes and current issues are in the 
stacks. 

d. None of the above 

34. in Hornbake Library, periodicals are arranged like this: 

a. Bound volumes are in the stacks; current issues are 
in the Periodical Room. 

b. Both bound volumes and current issues are in the 
Periodical Room. ^. 

c. Both bound volumes and current issues are in the 

stacks . 

d. None of the above 

35. When you can't find a periodical or an article using the 
tools in a library, you should: 

a. Give up and go home 

b. Atk another student 

c. Complain to your teacher 

d. Aik your friendly Reference Librarian 
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MAJIK lOHHATIVl IVALOATIOH 
F06TTI&T 



6tod«nt ID «: 



1. A c^nml periodical indmx includes: 
«. Popular Mffasinos 

b. ProfosBional jbomals 
o. Both of tha above 
d. Meitber of the above 

2. A Epeclaliaed periodical ladex Includes: 

a. Popular Mfasines 

b. Professional Journals 
o. Both of the above 

d. Neither of the above 

S. Which of these are kinds of periodical indexes? 

a. Newspaper indexes 

b. Haiatiae indexes 

c. Neither of the above 

d. Both of the above 

4 What is a periodical index? ^.4^ 
a A set Sf numbers •easurinf periodic activity 

b. A guide to the contents of periodicals 

c. An orcanised group of periodicals 

d. None of the above 

5 Periodical indexes are available in these foms: 

a. Print 

b. Microfon 
o. lleotrcnic 

d. All of the above 

6-8. Naae three periodical indexes. 

a. , — — — ' — ■ 

b. 

o. „■ — ■ • — — 
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9. Hhich .ubj#ct li#«dliig(») would yott wltct to find articlM 
on tho Untwsity of Mwlaad? 

b- Acadsuic •chiovoii#iit— »ocona*ry 
c- 6port»— oollofiato 
d. Hono of th% abovo 

10-19 Match tht lottors on tho illustration with the correct 
ni»o» on tho lilt boloH the illustretion. 



MIm N«t> Antrfct 



5lpr uto- 
Atcotel and lodim (Amencan) 
dhuilla todim 
Chippewa bidiam 
Crce Indiana 

KtHgiM ani aiytlMloay C 

See §ko 
Totem poks 

cL^^Ainnniiia native apirituatity: a all to justice fcovcr 
atorvj J. Matnuson. il Christ Cmiury 104:1114*17.0 

±3 Ve '-T — ' / 



\ 



20. 




( ) citation 

( ) subject b#«dln« 

( ) sttbh*adinf 

( ) voluB« nuBb«r 

( ) data of publioation 

( ) author 

( ) artiolo titla 

( ) Journal titX* 

( ) pa<e nuabcra 

( ) croaa rafar«noo 

Which indax would rou uao llxii to find articles on achool 

f inaneo? . , , ^ 

a. gutlrf«« P^rlftdloala Ind«t 

o. y<^-feiatloi>^^"" 
d. 



TK* itordB "aaa alao" In m indax toll you: 
a ThTindoI haa So ortlcloa at all about thia topic. 
5: SI iSdS hi. So nrticlo. fbout th. topio horo but ha. 
ralatod articloa undor another hoadiag . 

c. So iSdaS has artiolo. horo ud undor another hei^ding. 

d. None of the abovo. 
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22. Ih. !!or*.-~.:A«i5!ff»!tl!«S?' 



d. Hon* «f tl», «lbc>»«. :'.*^* ■ 



2}-26. 




( ) ladMc 
( ) frMM 

( ) IdMtifying l«tt«r 

26. In which fxfmm in th« illuitratlon bolow iwuld you find th« 
iournftl ftfitnt.**^ Pft^iiry? 



tCKHTIFiC NOtTCUifUKC 

tCCRTtK MCMKMlCAtCHITECTS AND MliAEtS COIT 

tcloiiitBi^^N^ ^ MSTirvTf of pnvsicm. and 




tMiffiiNMCftV 



27. 



b. tinm^ 

d. jtof^iif :«» •btw* 

To fina tho ffl i rn M l ftf Byfl^il M ^ ^'^l;" I?gS°^i?S SiS* 

— T! Li1>rinriM» 7C i wcwld IfK* on » «iorof ioho with wMon 
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28 What should you uso to loam which IIMCP llbranr or llbrwrlos 
ba^ a pMriodloel roa aood? 
«. OHCP 8ori«ls List 
b. A yorioiioal iadwt 

C. A 0«B(pUB MP 

d. OIAC, tho CMiptts oallao o«t«lof 



29-32. 



Kftteh tho lottoT* oo tho illustration with the corroot 
nuMiB bolow tho lllustrotioa. 




MODERN TEXTILES 

nCKELOlN LIBRARY 
b ^STACKS: V. 50(lW9)-62(Ho. 1-9.1981) 




34. 



35. 



C^UNOERGRADUATE LIBRARY 

PER. STACKS: V. 501 1969)-54( 1973). 
KR. STACKS: «ICROFIL«-*d 

V.55(1974)-62(t98t) 



( ) coll iiuabor 

( ) llbr&rr whoro periodical can bo found 

( ) part of llbrarjr ^ro periodical can bo found 

( ) f omat of periodical 

In Bombako Library, perlodlcale aro arranged like this: 

in «°J^«»~;^;%5Ji.7,^ the atacke; current Issues are 

In the Periodical Boca, 
b. Both bound volumes and current Issues are m ^ne 

Periodical Boo«. . 
o. Both bound voIubob and current Issues are in tne 

stacks . 
d. None of the above 

In hcKeldln Llbrarr, periodicals are arranged l";*.*^*; 

s. Bound voluaes aro in the stacks; current issues are 
la th« Perlddloal Boca. _ 

b. Both bound treluaes and current issues are in tne 
Periodical Boom. . 

c. Botb^owad volumes and current Issues are in tne 

staow- 

d. Noiia of the above 

Hhen you oan't find a porlodloal or an article using the 
tools in a library* you should: 
s. Oivo up and go hoMS 
b. Ask another student 

e. Coaplain to your teacher 

d. Ask your friendly Beforence Librarian 
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EXHIBITS 5-10 
EXEMPLARY SCREENS 
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F.xhibit 5 




.J 



,f you are jusl beginning to use periodical articles for research, 
start with the section called Periodical Indexes. 



You will learn how to: 



•choose a periodical IndcK . 
•choose appropriate subject headings, 
•identify the ittms within each citation, 
•find periodical titles on the UMCP Serials List, 

•locate periodicals within the UMCP libraries 

System. 
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Exhibit 7 



The right specialized index con leod you to the kinds of articles you need. 
The wrong it^dex ccn lead you on o wiid goose chase! 
Here ore \he names of the most useful indexes, both ^ v . 
general and specialized 

• Bioiogico! and Bgricuiturai indBH 

• Business Feriodicafs /rnf en 
« Biamondback IttdeH 

• Education inden 

• Beltarai Science inden n«no 

More names on next page. 

^- ^ 

StriJlJlist Vl<«r» to go 




5. Safety in engineering design and construction 

a. National NBUtspapBrlndBH 

b. firt ladeH 

c ftppliFd Science and Technology liyfUOM 
(gi Click on the button for the right answer. 

6. Microcomputer applications in business 
a Business Periodicals IndoM 

b. pppiled Science and Teclmology laden 

c. Reader's eulde to Periodical ilteratutv 
^ Click on the button for the right answer. 




ft.ri.l« Urt Vhfr* to < 



■TOT 



IS 

■iERJC 
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YOU have finished the introducUon 

TO go to the section on Periodical Indexes click on the p button. 
TO go to the section on the UMCP Serials List click on theH button. 
TO go to the section on Where to Find Periodicals click on the^button. 




professional Journals • 

Newspaper Indexes list newspaper articles by subject for a particular 
time period. For exannple: 

NdtiMdl Newspaper index 



inwii 



♦ New Vork Times Index 

• Diomondbock Index 





mi. 
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EXHIBIT 11 



UNIVERSI'n OF MARYLAND 
Collegs Park, MD 20742 

INSTITUTIONAL REVIEW BOARD 
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PRINCIPAL INVESTIGATOR: 
(or Faculty Advisor) 



Dr. H. Delia Neuman 



STUDENT INVESTIGATOR: 
PROJECT TITLE: 



Hypertext Introduction to UMCP Serials List 



The Chairman of the University IRB reviewed the above-mentioned project 

on 3,1-89 in accordance with P\iblic Health Service grant policy 

as defined in "The Institutional Guide to DHHS Policy on Protection of Human 
Subjects," 12-2-71, and in Title 45, Code of Federal Regulations, Part 46, 
and found this project to be exen?)t. 



Exemption No. 



Chairman of Institutional Review Board 



The Principal Investigator and Student Investigator, in signing this 
report, agree to notify the Office of the Dean for Graduate Studies and 
Research of any addition to or changes in procedure subsequent to this 
review. 



Principal Investigator 
(or Faculty Advisor) 



Student Investigator 



mjtaag BRrmai Qua SIGHKD oopy K) TOE «o>nn*'rB gnmnt., bm 2133. aooTO 
APtllKISTRATIOH BOILDING 



4/88 
IRB-4 
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HOW TO WORK riAJIK 



/L^S^ITfImd journal instructional KIT) IS A SELF-GUIDED INSTRUCTIONAL 



MODULE. 



"> BOLDFACED WORDS OR PHRASES 

3 ARROWS ON THE LOWER RIGHT CORNER OF THE SCREEN 

AT THE BEGINNING OF THE MAJIK MODULE. 
HOW TO START. 

/. WHEN YOU SIT DOWN AT THE COMPUTER, YOU WILL SEE THE TITLE SCREEN OF MAJIK. 
2. IKTHELOWERRIGHTCORNEROFTHESCREENYOUWILLSEEANARRO^'POINTINGTOTHE 
RIGHT. 

J. US.NGTHE f^USE, MOVETHE CU^^f/jSM^^^^^^^^^^^ HNGER) DOWN 

TO THE ARROW. MAKF SURE THE FINGER IS POINTING TO THE ARROW. 

4. CLICKTHE MOUSE ONCE. YOU SHOULD SEE THE NEXT SCREEN OF THE MODULE THAT 
EXPLAINS THE ARROWS. 

c munvf TO THE NEXT SCREEN WHICH EXPLAINS THE ICONS, 00 AS YOU DID IN STEPS 3 
6 TO GO BACK ONE SCREEN, USE THE ARROW POINTI NG TO THE uEFT. 

fuSnWER WIKTIsTtO THE BUnON VOU WANT m THEN CLICK THE MOUSE ONCE. 



EXHIBIT 13 



MAJIK FORMATIVE EVALUATION: OBSERVATION PROTOCOL 



Student name: 



Rsrchr. initials: „ Date £c time: 

Time required to complete program: 



section 1: Introduction to Periodical Indexes 
Selected? 

Strengths/weaknesses : 

Reading/understanding : 

Navigating: 
Graphics : 

Practice exercises: 
Strategies: 

Student comments/suggestions: 



Researcher comments/suggestions: 



section 2: Using Periodical Indexes 
Selected? 

Strengths/weaknesses : 

Reading/understanding : 

Navi9ating: 
Graphics : 

Practice exercises: 
Strategies : 

Student comments/suggestions : 
ERIC Researcher comments/suggestions: 
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section 3: Using the UMCP Serials List 
Selected? 

Strengths/weaknesses : 

Reading/understanding : 

Navigating: 

Graphics: 

Practice exercises: 
Strategies: 

Student comments/suggestions: 
Researcher comments/suggestions : 

section 4: UMCP Libraries 
Selected? 

Strengths/weaknesses : 

Reading/understanding : 
Navigating: 
Graphics : 

Practice exercises: 
Strategies: 

Student coirjients/suggestions : 
Rt;searcher comments /suggest ions : 

SUMMARY: 



JC DH/6-89 
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Section 

Using Per Indexes 
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EXHIBIT 14 
SAMPLE DATA ANALYSIS MATRIX 



38 min. 
Non-Mac User 
[Name] 
1 



19 min. 
Mac User 
[Name] 

2 



35 min. 
Non-Mac User 
[Name] 

3' 



R « reading 

N « navigation 

G » general 

PE « practice exercises 

• revision information 



R: no problem 
N: linear 



G: did not remember 
boldface was an option 

PE: no problem, but did not 
like buttons; wanted 
the boldface to indicate 
rJ t answers 



R: no problem 

N: nonlinear 
used 1 cons* rather than 
TOC; was able to find 
way in and out of sections 

G: '•good'* 



R: no problem 
N: linear 



G: button-letter 
problem; unclear how 
to go about 



PE: '•helped to remember terms PE: best section; 

reched anwers 
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EXHIBIT 15 

SELECTED STUDENTS* SUMMARY COMMENTS 



Graphics good withm the limits of the Mac. 

Graphics were very good and fun; make you pay attention and 

move more intensely through the module. 
Entertaining graphics kept attention. 
Cute graphics. 
Excellent graphics. 
Good graphics. 



l£5t 

Nice, straightforward, easy to understand text. 
Text was very good. 
Large amount of text but OK. 
Information clear. 

Pl^i^licfe Exercises: 

Very good. 

Good; a check on knowledge. 
Helpful; good review of principles. 
Not hard; reinforcing. 
Learned through practice exercises. 

Self-Pacing: 

Let*s you go at your own pace. 

Movement 

See anything you want over again, at any time. 
Could always go back. 
I could go back easily. 



v4 
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